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SURVEILLANCE SYSTEMS AND METHODS 

BACKGROUND 

[0001] The present disclosure generally relates to surveillance systems and methods. 
In particular, the present disclosure relates to surveillance systems and methods that 
combine video and radio frequency identification. 

[0002] Shoplifting prevention and inventory control are becoming more important to 
many commercial retail stores as way to minimize loses. Surveillance systems and 
methods are often used to achieve the desired reduction in losses. 

[0003] Video surveillance systems are a common tool used m the efforts to prevent 
shoplifting and control inventory. Typical video surveillance systems use one or more 
cameras to survey an area. This allows a security officer to track a potential shoplifter 
through a shopping area, which in the line of sight of the camera. Unfortunately, such 
video surveillance systems alone have not proven effective at achieving the desired 
reductions in shophfting at an acceptable cost. 

[0004] Radio frequency identification (RFID) systems are also becoming 
commonplace in the efforts to prevent shoplifting and control inventory. 
Advantageously, RFID does not require direct contact or line-of-sight scanning as in 
video surveillance systems. RFID systems incorporate the use of a tag and a scanner. 
The tag can emit electromagnetic or electrostatic signal in the radio frequency (RF) 
portion of the electromagnetic spectrum. The tag can then be placed on an object, 
animal, or person to uniquely identify that item. The scanner can detect the presence 
or absence of the emitted signal. RFID is sometimes called dedicated short-range 
communication (DSRC) since the emitted signal can be detected by the scanner within 
about a one-meter radius. Accordingly, many retail outlets have installed scanners at 
the points of entry and/or exit and include the tag on a piece of merchandise. In this 
manner, any merchandise having an active RFID tag will be detected as the item 
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passes the scanner. The retail outlet can selectively deactivate and/or remove the tag 
of items that are approved to exit the area, such as those purchased by a customer. 
Unfortunately, such RFID systems alone have also not proven effective at achieving 
the desired reductions in shoplifting at an acceptable cost. 

[0005] Accordingly, there is a continuing need for surveillance systems and methods 
that overcome and/or mitigate one or more of the aforementioned and other 
deficiencies and deleterious effects of prior systems and methods. 

SUMMARY 

[0006] A surveillance system having a video subsystem, a radio frequency 
identification subsystem, and a processor is provided. The video subsystem detects a 
video image of a tagged item. The radio frequency identification subsystem detects a 
position of the tagged item. The processor communicates with the video and radio 
frequency subsystems to monitor a condition of the tagged item based at least in part 
on the video image and the position. 

[0007] A surveillance system having a first loop antenna, a second loop antenna, and 
a signal processor. The second loop antenna is substantially orthogonal to the first 
loop antenna. The first and second loop antennas are inductively couplable with a tag 
through magnetic fields. The signal processor estimates an orientation of said tag 
based on the magnitude of the inductively coupled modulated signal from the tag as 
the orientation of the coupling field generated from the first and second loop antennas 
is scanned through a range of angles. 

[0008] A surveillance method is also provided. The method includes determining an 
orientation of an RFID tag, determining a position of the RFID tag; and providing the 
orientation and the position to a video-processing component. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

[0009] These and other features, aspects, and advantages of the present disclosure will 
become better understood with reference to the following description, appended 
claims, and drawings where: 

[0010] FIG. 1 is a block diagram of an exemplary embodiment of a surveillance 
system; 

[0011] FIG. 2 is a block diagram of relevant portions of a radio frequency 
identification subsystem; 

[0012] FIG. 3 is an illustration of an example orientation measurement; 

[0013] FIG. 4 is an illustration of example control signals for the RFID subsystem of 
FIG. 2; 

[0014] FIG. 5 is a flow chart of a first exemplary embodiment of a surveillance 
method; and 

[0015] FIG. 6 is a flow chart of an altemate exemplary embodiment of a surveillance 
method. 

DETAILED DESCRIPTION 

[0016] Referring now to FIG. 1, an exemplary embodiment of a surveillance system 
10 in use in a surveillance area 12 is illustrated. System 10 includes a processor 14 
for integrating a radio frequency identification (RFID) subsystem 16 and a video 
subsystem 18. System 10 integrates subsystems 16, 18 to track and provide 
information about an RFID tagged item 20 within area 12. 
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[0017] RFID subsystem 16 includes a number or plurality of scanners 22 at 
predetermined locations within area 12. Similarly, video subsystem 18 includes a 
number or plurality of cameras 24 at predetermined locations within area 12. 
Scanners 22 and cameras 24 are in electrical communication with processor 14 so that 
surveillance system 10 can integrate data received from the scanners and cameras to 
provide enhanced surveillance of item 20, 

[0018] Scanners 22 can detect RFID tagged item 20 when the item is within about a 
one-meter radius. Cameras 24 can detect RFID tagged item 20 when it is within the 
field of view of one of the cameras. Advantageously, surveillance system 10 is 
configured to track tagged item 20 using scanners 22 when with the detection range of 
the scanners, but using cameras 24 when with the field of view of the cameras. 
System 10 can automatically switch its surveillance of tagged item 20 between 
scanners 22 and cameras 24 within area 12 as the item is moved throughout the area. 
In this manner, system 10 can determine the position of tagged item 20 within a 
surveillance area 12. 

[0019] It has been determined that it can be more difficult to discriminate between 
casual behavior and theft with only position information. Without knowing the 
orientation of tagged item 20, it can be difficult to recognize desired information 
about the tagged item when viewed by cameras 24. Accordingly, system 10 is 
configured to determine the orientation of tagged item 20 within surveillance area 12. 

[0020] For example, some customer behavior with respect to tagged item 20 cannot 
be detected under certain conditions of system 10, such as when the tagged item is 
partially outside the field of cameras 24. It has been found that combining the 
position and orientation of tagged item 20 from RFID subsystem 16 with the video 
from video subsystem 18 allows surveillance system 10 to efficiently predict an 
expected appearance of the tagged item. Here, surveillance system 10 can then 
compare the expected appearance to an actual video image to detect authorized 
tampering. 
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[0021] Referring now to FIG, 2, an exemplary embodiment of scanner 22 of RFID 
subsystem 16 is illustrated. Scanner 22 includes three loop antennas 100 are arranged 
substantially orthogonal to one another. 

[0022] Loop antennas 100 are driven by a z ramp generator 104, a y ramp generator 
106, and an x ramp generator 108, respectively, through voltage variable attenuators 
1 10 and amplifiers 1 12. Antennas 100 receive signals indicative of tagged item 20. 
These received signals are summed through a splitter 114 and are sent a receive chain 
116. 

[0023] It should be recognized that scanner 22 is illustrated by way of example having 
three loop antennas 100. Of course, it is contemplated by the present disclosure for 
each scaimer 22 to have less than three loop anteimas 100. For example, it is 
contemplated for scanner 22 to have two loop antennas 100. 

[0024] Loop antennas 100 can be any loop antenna, such as the Texas Instruments 
(TI) Series 6000 Gate Antenna RI-ANT-TOl. This TI gate antenna is used with 
readers having a transmitter fi-equency of 13.56 MHz and an output impedance of 50 
Ohm, such as the TI S6500/ 6550 Readers. 

[0025] Tagged item 20, in this example, includes an inductive passive tag capable of 
being read by loop antennas 100 when the tagged item 20 is within an interrogation 
zone 102 of scanner 20. Interrogation zone 102 can be about one meter in each 
direction from loop antennas 100. ISO Standard 15693-2, a communications protocol, 
defines one method for reading data from inductive passive tags. In this example, 
tagged item 20 is inductively coupled with magnetic fields through loop antennas 100. 

[0026] Z ramp generator 104, y ramp generator 106, and x ramp generator 108 control 
the amplitude of the 13.56 MHz RF antenna excitation waveform for antennas 100 by 
way of a ramp waveform. For example, ramp generator 104, 106, 108 can be the 
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Agilent Technologies 33220A Function / Arbitrary Waveform Generator. Attenuator 
110 is a device for reducing the amplitude of an AC wave without introducing 
appreciable distortion. Amplifier 1 12 is an electronic device that increases the 
voltage, current, and/or power of a signal Splitter 114 is a device that divides a signal 
into two or more signals, each carrying a selected frequency range, or reassembles 
signals from multiple signal sources into a single signal An example of splitter 1 14 is 
Mini-Circuit's power splitter ZSC-2-1. 

[0027] Receive chain 1 16 is a signal processing component that includes, for 
example, a bandpass filter 1 18, an envelope detector 120, a modulation minimum 
(null) detector 122, and an angle calculator 124. In some embodiments, there is a 
receive chain for each loop antenna 100. The resulting orientation calculated by angle 
calculator 124 is provided to a video processing component 126. 

[0028] Bandpass filter 1 18 is an electronic device or circuit that allows signals 
between two specific frequencies to pass, but that discriminates against signals at 
other frequencies. An example bandpass filer 1 18 has a filter passband of 13.98375 
MHz ± 50 KHz. Envelope detector 120 detects the envelope (upper and lower 
bounds) of the waveform as described in detail below with respect to FIG. 4. 

[0029] Modulation minimum detector 122 finds the point at which the envelope is at 
a minimum (null). The tag modulation minimum indicates a magnetic field is at 
substantially right angles to tagged item 20. Angle calculator 124 determines the 
orientation of tagged item 20. At certain times during the antenna excitation, the 
magnitude of the tag modulation signal as received by a single antenna can be 
measured. The measurements for the X, Y, and Z antenna can be used with the 
orientation of the tagged item to determine the position of the tagged item in the 
interrogation zone of the antenna. 

[0030] Video processing component 126 is provided by video subsystem 18 to 
processor 14, Video subsystem 18 is any video system capable of tracking tagged 
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item 20, such as merchandise, in area 12. In one embodiment, video processing 
component 126 comprises a tracking mechanism, an object verification mechanism, 
and a recognition mechanism. The tracking mechanism tracks people and objects. 
The object verification mechanism verifies tag information with video images. The 
recognition mechanism recognizes patterns in the video images. 

[0031] After receiving the orientation firom RFID subsystem 16, processor 14 is able 
to use the orientation of tagged item 20 to compare the video image of the object to 
the expected appearance of the object at that orientation. As a result, some tampering 
and shoplifting is detectable. 

[0032] Processor 14 can communicate with subsystems 16, 18 by known 
communication methods such as, but not limited to, as Ethernet. Here, video 
processing component 126 includes software components, such as segmentation 
routines, temporal association routines, geometric reconstruction routines, RFID 
object detection, RFID position and orientation detection, object tracking, person 
tracking, behavior analysis, probabilistic engines, and Bayesian frameworks. 

[0033] In some embodiments, video processing component 126 activates RFID 
subsystem 16 when a person is within interrogation zone 102 of scanner 20. If a 
person is in zone 102 with tagged item 20, the person changes the magnetic coupling 
between tagged item 20 and loop antennas 100 by virtue of their body being present in 
the magnetic field. System 10 is configured to detect these changes the magnetic 
coupling between tagged item 20 and loop antennas 100 by virtue of their body being 
present in the magnetic field. 

[0034] A 13.56 MHz clock signal 130, and other clock signals 128 are included in 
example subsystem 16, which has a frequency of 13.56 MHz. In some embodiments 
one or more of clock signals 128 is a frame rate clock from video processing 
component 126. 
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[0035] Referring now to FIG. 3, an exemplary orientation and position measurements 
relative to three axes is illustrated. The orientation is calculated by angle calculator 
124. The orientation is a triple, (O, a, 9) where O (phi) 200 is the angle measured 
from the z-axis 202 to the y-axis 204, 8 (theta) 206 is the angle measured from the y- 
axis 204 to the x-axis 208, and a (alpha) 210 is the angle measured from the x-axis 
208 to the z-axis 202. In some embodiments, the orientation is a single angle relative 
to two axes. 

[0036] FIG. 4 shows example control signals in six rows for subsj^tem 16. The first 
row 300 shows a clock signal. The second row 302 shows a signal from x ramp 
generator 108. The third row 304 shows a signal from y ramp generator 106. The 
fourth row 306 shows a signal from z ramp generator 108. The fifth row 308 shows 
an interrogation field angle from loop antennas 100 varying between about 0 and 180 
degrees. In practice, the angle is not swept linearly, but during a calibration phase the 
field is measured to correct for nonlinearities in time and space. These corrections can 
be used to modify the ramp signals to produce a magnetic field angle that sweeps 
linearly with time. The sixth and last row 310 shows a bandpass filter / envelope 
detector output (tag modulation signal). 

[0037] In some embodiments, the clock signal in first row 300 is a frame rate clock 
from video processing component 126. In some embodiments, the clock signal is 
dependent on how long it takes to read tagged item 20. 

[0038] At the start of the first clock period, x ramp generator 108 is at fixU power, y 
ramp generator 106 is at zero, and z ramp generator 104 is at zero. Under these 
conditions, loop antenna 100 in the x-direction is excited and a x-amplitude tag 
modulation signal 312 (shown in row six 3 10) is read from receive chain 1 1 6. The x- 
amplitude of the inductive signal is used to correct for the x, y, and z offset and to get 
the x-coordinate of the position (x, y, z) of Tagged item 20. 
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[0039] About in the middle of the first clock period, y ramp generator 106 is at full 
power, X ramp generator 108 is at zero, and z ramp generator 104 is at zero. Under 
these conditions, loop antenna 100 in the y-direction is excited and a y-amplitude tag 
modulation signal 314 (shown in row six 310) is read from receive chain 116. The y- 
amplitude of the inductive signal is used to correct for the x, y, and z offset and to get 
the y-coordinate of the position (x, y, z) of tagged item 20. 

[0040] About in the middle of the second clock period, z ramp generator 104 is at full 
power, X ramp generator 108 is at zero, and y ramp generator 106 is at zero. Under 
these conditions, loop antenna 100 in the z-direction is excited and a z-amplitude tag 
modulation signal 316 (shown in row six 310) is read from receive chain 116. The z- 
amplitude of the inductive signal is used to correct for the x, y, and z offset and to get 
the z-coordinate of the position (x, y, z) of Tagged item 20. hi some embodiments, 
the X-, y-, and z-coordinates are all read within one clock period, or about the time it 
takes to read tagged item 20. 

[0041] As shown in row five 308, each angle in the orientation (O, a, 0) is calculated 
at a tag modulation minimum (null) 3 1 8 (shown in row six 310). Angle O (phi) 200 is 
calculated, when an x-antenna signal is zero and tag modulation minimum 318 occurs. 
Angle 9 (theta) 206 is calculated, when a z-antenna signal is zero and tag modulation 
minimum 318 occurs. Angle a (alpha) 210 is calculated, when a y-antenna signal is 
zero and tag modulation minimum 318 occurs. 

[0042] FIG. 5 is a flow chart of an example surveillance method. In step 400, the 
orientation of tagged item 20 is determined with respect to three mutually orthogonal 
planes using inductance and three mutually orthogonal antenna loops. For example, O 
(phi) 200, 0 (theta) 206, and a (alpha) 210 are determined with respect to the x-y, y-z, 
and z-x planes, as shown in FIG 3. In step 402, the position of tagged item 20 is 
determined with respect to the three mutually orthogonal planes. For example, the x-, 
y-, and z-coordinates of the position (x, y, z) are determined at tag modulation 
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minimum 318, as shown in row six 310 of FIG. 4. In step 404, the orientation and 
position of tagged item 20 is provided to a video-processing component. 

[0043] FIG. 6 is a flow chart of another example surveillance method. In step 500, 
the orientation and position of tagged item 20 as a function of time is received from an 
RFID subsystem. In step 502, a person and an object with an RFE) tag are tracked via 
a video subsystem. In step 504, an alert is provided indicating that the person 
acquired the object without purchasing it. 

[0044] For example, suppose video processing component 126 recognizes a person 
stopping in front of a table displaying tagged item 20. Based on the video information 
from subsystem 18, RFID subsystem 16 is activated. As the person interacts with the 
tagged item 20, system 10 tracks hand motions, face motions, RFID information of 
the object and the like. If the person picks up the item, video subsystem 18 tracks the 
person within area 12 to estabhsh whether the person placed the item down. For 
example, video subsystem 18 analyses the tracking of the person and the object, 
including object recognition and RFID. If the item was not placed anywhere then a 
strong hypothesis is built based on the interaction that the person still has the item. If 
so, a real-time alert is produced and a synopsis is provided, including salient video 
clips. In addition, a history of the person and object tracking is available. 

[0045] Orientation information provided by RFID subsystem 16 to surveillance 
system 10 aids in analysis. For example, system 10 can analyzes events, such as 
whether the object was placed in a shopping cart or handled in a secretive fashion 
using inputs from subsystems 16, 18. Another example is analyzing the appearance of 
tagged item 20. Here, surveillance system 10 can generate a synthesized appearance 
of the tagged item at the orientation provided by RFID subsystem 16. Surveillance 
system 10 can then compare the synthesized appearance with the actual appearance 
provided by video subsystem 18 to determine whether tagged item has been altered 
(e.g., authorized tampering). 
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[0046] While the present disclosure has been described with reference to one or more 
exemplary embodiments, it will be understood by those skilled in the art that various 
changes may be made and equivalents may be substituted for elements thereof without 
departing from the scope of the present disclosure. In addition, many modifications 
may be made to adapt a particular situation or material to the teachings of the 
disclosure without departing from the scope thereof Therefore, it is intended that the 
present disclosure not be limited to the particular embodiment(s) disclosed as the best 
mode contemplated, but that the disclosure will include all embodiments falling 
within the scope of the appended claims. 
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